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£9^^ fe((xerms of art appear frequently in the following. For ease of reference tliey are 



defined here as follows: 

10 "Contend refers to multimedia content This term encompasses tiie various tj'pes of 

infomiation to be processed in a multimedia entertainment system. Content specifically 
refers to digitzed audio, video or still images in the context of this discussion. This 
information may be contained within files on a multimedia computer system, the files 
having a particular fomiat specific to the modality of the content (sound, images, moving 

15 pictures) or the type of systems, computer or otherwise, used to process the content. 

^'Digitized" refers to content composed of discrete digital samples of an otherwise analog 
media, which approximate that media inside a computer or othCT digital device. For 
instance, tlie sound of music occurs naturally, and is experienced by humans as an analog 

20 (continuous) sound wave. The soimd can be digitized into a stream of discrete samples, or 
numbers, each of which represents an approximate value of the amphtude of tlie real 
analog wave at a particular instant in time. These samples can be stored in files in a 
computer and then used to recreate the original sound wave to a high degree of accuracy. 
In general, content entering a digital system is digitized by Analog to Digital converters 

25 (A/D) and analog media are recreated by the digital system using a Digital to Analog (D/A) 
converter. In the context of this discussion content is always digitized content 

** Glyptography" is a field coveiing numerous teclmiques for scrambling infonnation 
conveying messages so that when tlie message is conveyed between tlie sendei' and 
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reciever an unintended party who intercepts tliis message cannot read it, or extract useful 
information from it 

A "Public Key Cryptosystem" is a particulai' cryptographic system where all parties 
posses pairs of keys for encryption and decryption. Parties to this type of system freely 
distribute their public keys, which other may use to encrypt messages to the owner of tlie 
public key. Such messages are decrypted by the reciev^ with the private key. Private keys 
aie nevei- distiibuted. A message encrypted with a public key can only be decrypted with 
the corresponding private key, and vice versa. A message ^crypted with a private key is 
said to have been signed by the owner of that key. Anyone in possession of the public key 
may decrypt tlie message and know that it was encrypted, and thus signed, by the ownei' of 
tlie public key, since only they possess the corresponding private key. 

'*Steganography" is a field distinguished from cryptography, but associated with it, that 
covers numerous methods for hiding an informational message within some other 
medium, perhaps another imrelated message, in such a manner that an unintended party 
who intercepts the medium cany'ing tlie hidden message does not know it contains this 
hidden message and therefore does not obtain the information in the hidden message. In 
other words, steganography seeks to hide messages in plain view. 



Background of the In vqition 



In the current environment of computer networks and the proUferation of digital or 
digitized multimedia content which may be distributed over such networks, a key issue is 
copyright protection. Copyright protection is the ability to prevent or deter the proliferatioc 
of unautiiorized copies of copyrighted works. It provides a reasonable guarantee that the 
author of a copyrighted work will be paid for each copy of that work. 

A fundamental problan in the digital world, as opposed to the world of physical media, is 
tliat a imlimited number of perfea copies may be made from any piece of digital or 
digitized content A pafect copy means tliat if the original is comprised of a given sti-eam 
of numbers, then the copy matches the original, exactly, for each numbo" in the stream. 
Thus, tliere is no degradation of the original signal during the copy operation. In an analog 
copy, random noise is always introduced, degrading the copied signal. 

The act of making unlicensed copies of some content, digital or analog, whether audio, 
video, software or other, is gmeralty known as piracy. Piracy has been committed for the 
purpose of either profit from the sale of such unlicensed copies, or to procure for the 
"pirate" a copy of the contrat for personal use without having paid for it 

The problem of piracy has been made much worse for any type of content by the 
digitization of content Once content enters the digital domain, an unlimited number of 
copies may be made wiUiout any degradation, if a pirate fuids a way to break whatever 
protection scheme was established to guaid against such abuses, if any. In tlie analog 



5 world, there is generally a degradaUon in the content (signal) with each successive copy, 
imposing a sort of natural limit on volume of piracj'. 

To date, three general types of schemes have been implemented in an attempt to protect 
copyrights. 
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1) Encryption 

2) Copy Protection 

3) Content Extensions 



15 Copy Protection and Content Extensions generaUy apply in the digital world only, while 
scheme related to Encryption, commonly known as sci^bling, my be applied to an 
analog signal. This is t)'pical in analog cable systems. 
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Encnrtion scrambles the content Before the content is made ready for deliver,, whether 
on floppy disk, or over a network, it must be enoyp^d, or scmnbled. Once the content 
has been enc^'pted. itcamiotbe used untU it is deaypted. or unscrambled. Enciypted audio 
data might sound like incomprehensible scre^hing. while an encrypted picture or video 
might appear as random patterns on a screen. The principle of encryption is that you are 
free to make as many copies as you want, but you can't read anything that makes sense 
25 until you use a special key to decrypt, and you can only obtain the key by paying for the 



content. 



Encryption has two problems, however. 1) Pirates have historically found ways to crack 
encryption, in effect, obtaining the key without having paid for it; and 2) Once a single 
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5 legitimate copy of some content has been decrypted, a pirate is now free to make unlimited 
copies of the decrypted copy. In effect, in order to sell an unlimited quantity of an 
encrypted piece of software, the pirate could simply buy one copy, which tliey are entitled 
to decrypt 

10 Copy Protection includes various methods by which a software engineer can write the 

software in a clever manner to determine if it has been copied, and if so to deactivate itself. 
Also included are undocumented changes to tlie storage format of the content Copy 
protection was generally abandoned by the software industr>^ since pirates were generally 
just as clever as the software engineers and figured out ways to modify their software and 

1 5 deacti\'ate the protection. The cost of developing such protection was not justified 
considering tlie level of piracy which occurred despite the copy protection. 

Content Extmsion refers to any system which attaches some extra information to the 
original content which indicates whether or not a copy may be made. A software or 

20 hardware system must be specifically built around tiiis scheme to recognize the additional 
infomiation and interpret it in an appropriate manner. An example of such a system is the 
Serial Copyright Management System embedded in Digital Audio Tape (DAT) hardware. 
Under this system, additional information is stored on tlie disc immediately preceding each 
track of audio content which indicates whether or not it can be copied. The hardware reads 

Z5 this information and uses it accordingly. 

A fundamental problmi with Encryption and Content Extension is the "rogue engineer". 
An employee who helped design such a system or an individual witli tlie knowledge and 
means to analyze such a systan can modify it to ignore the copyright infomiation 
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altogether, and make unlicensed copies of the content. Cable piracy is quite common, aided 
by illicit decoder devices built by ttiose who understand the technical details of the cable 
encryption system. Although the cable systems in question were actually based on analog 
RF signals, the same principle appUes to digital systems. 

Tlie practical considerations of weak encryption schemes and rogue engineers have ser\'ed 
to limit the faith which may be put in such copyright protection schemes. The invention 
disclosed herein serves to address these problems with conventional systems for digital 
distribution. It provides a way to enforce copyright online. TTie invention draws on 
techniques from two fields, cryptography, flie art of scrambling messages so that onfy the 
intended recipient may read them, and steganogr^hy, a term appUed to various techniques 
for obscuring messages so that onfy the mtended parties to a message even know that a 
message has been seat, thus it is termed herein as a stega-cipher. The stega-cipher is so 
named because it uses the steganographic technique of hiding a message in multimedia 
content, in combination with mult^le keys, a concept originating in cryptogr^hy. 
However, instead of using the keys to ^crypt the content, the stega-cipher uses these keys 
to locate tlie hidden message witliin the content The message itself is encr>'pted which 
serves to further proctect the message, verify tlie validitj' of the message, and redistribute 
the iaformation in a random manner so that anyone attempting to locate the message 
without the keys cannot rely on pre-supposed knowledge of the message contents as a help 
in locating it 
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Suniniar}' of the Invention 
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IS 



The invention disclosed herein combines two techniques, steganography - obscuring 
iiifomiation that is otlierwise in plain sight, and cryptography - scrambling infomiation tliat 
must be sent over unsecured means, in a manner such that only the intended recipiem may 
successfully unscramble it. The net effect of this system is to specifically watennark a 
piece of content so that if it is copied, it is possible to detennine who owned the original 
from which the copies were made, and hence determine responsibility for the copies. It : 
also a feature of the system to uniquely identify the content to which it is appUed. 



15 For a comprehensive discussion of cryptography, its theory, applications and specific 
algorithms, see APPUED CRYPTOGRAPHY, by Bruce Sdmeier, which is herein 
incorporated by reference at pages 66-68, 387-392. 

Steganography is discussed briefly in THE CODE BREAKERS by David Kalm. which is 
20 herein incorporated by reference at pages xiii, 81-83. 522-526. and 873. An example 

application. Stego by Romana Machado, is also available for die Apple Macintosh. Stego 
can be found at the internet uniform resource locator "ftp://sumex-aim.stanford.edu/,nfo- 
mac/cmp/stegol Oa2.hqx". This application demonstrates a simple steganographic technique 
to encode a text message into a graphical image without significantly distorting the image. 



Tlie invention improves upon the prior art by providing a manner for protecting copyright 
in the digital domain, which neither steganography or cryptography does. It improves 
specificaUy on steganography by making use of special keys which dictate exactly where 
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5 within a larger chunk of content a message is to be hidden, and makes tlie task of exu-acting 
such a message without the proper key the equivalent of looking for a needle in a haystack. 

Hie information encoded by the Stega-Cipher process serves as a watemiaik wliich 
identifies individual copies of content legally licensed to specific parties. It is integral with 
10 tlie content It cannot be removed by omission in a transmission. It does not add any 
overhead to signal transmission or storage. It does allow the content to be stored to and 
used with traditional offline analog and digital media, without modification or significant 
signal degradatioa These aspects of the stega-cipher all represent improvements to tiie art 

15 The invention described herein is used for protecting and enforcing copyrights in the digital 
or on-line domain, where th^e are no physical limitations on copying copyrighted content 

The invention uniquely identifies everj' copy of multimedia content made using the 
invention, composed of digitized samples whetho" compressed or uncompressed, 
20 including but not limited to still digital images, digital audio, and digital video. 

The invention is for use in meterware or pay-by-use systems where an online user incurs a 
charge each time they access a particular piece of content, or uses a software title. 

25 Tlie invention is for use as a gai^al improvement to cryptographic techniques to increase 
tlie complexity of crj'tanalysis on a given cipher. 
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5 It is considered that the method and steps of the present invention will be modified lo 
account for the effects of lossy compression schemes on the samples and particularly 
includes modification to handle MPEG compressed audio and video. 

It is considered that statistical data spreading and recovery techniques, enor coding or 
10 spread spectrum processing techniques might be applied in the inv^tion to handle the 
effects of lossy compression, or counter the effects of a randomization attack. 

It is consideied that the apparatus desaribed might be further specialized and optimized in 
hardware by replacing gen^l purpose data buses and CPU or DSP driven operations with 
1 5 hardwired circuitry, incorporated in one or more special purpose puipose ICs. 

It is considered tliat the apparatus will be modeled and implemented in software on general 
purpose computer platforms. 

20 It is considered that stega-cipher hardware could be embedded in a consumei* electronics 

device and used to not only identify content and copyright, but to enable use of that content. 
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5 Detailed Description 

I. Digital Copyright Stega-Cipher Protocol and the Decode/Encode Program 

The purpose of the program described here is to watennark digital multimedia content for 
10 distribution to consuma^ through online services in such a way as to meet the following 
criteria 

Given a unique piece of multimedia content, composed of digitized samples, it is desirable 
to: 
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1) Uniquely identify this particular piece of content from others in a manner which is 
secure and undeniable (e.g. to know whether a digital audio recording is "My Way" by 
Frank Sinatra, or "Stairway to Heaven", by Led Zeppelin), and in a manner such that this 
Identification can be perfonned automatically by an electronic device or mechanism. 

2) Uniquely idenUfy tlie copyright owner of tlie content, and the terms under which it may 
be distributed in general, in a manner which is secure and undeniable. 



3) At such time as is necessary, additionaUy, uniquely identify in a secure and undeniable 
2 5 manner tlie licensed publisher who received a particular copy of the content, and the terms 
under which they may redistribute or resell it. 
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4) At such time as is necessary, additionally, uniquely identify in a secuie and undeniable 
manner, the licensed subscriber who received a particular copy of the content from the 
publisher described in item 3. 

Tlie program described in more detail below combines ttie techniques of crytography and 
steganography to hide a securely encrypted digital copyright certificate which contains 
information satisfying the criteria listed above, in such a manner as to be integral with the 
content, like a watermark on paper, so that possession of the content dictates possesion of 
tlie watermark information. In addition, the watermark cannot be "found" or successfully 
decoded, without possesion of the correct "masks" or keys, available only to those 
legitimately authorized, namely, those parties to a commercial transaction involving the sale 
of a copy of the content Finally, the ability to distribute such watermarked content in a 
system which implements the watermark scheme is denied without a successfully decoded 
watermark. Because well known and tested cryptographic techniques are used to protect the 
certificate itself, these certificates are virtually impossible to forge. 

Tlie basic program represents a key part of the invention itself. This program is tlien used 
as the metliod by which copyright information is to be associated in an integral manner 
witli the content This is a concept absent from copy protection, encryption and content 
extension schemes. The copyright information itself can be made undeniable and 
unforgeable using cryptographic techniques, so tliat through it an audit trail of ownership 
my be established for each copy of a given piece of content thus customizing each copy to 
a particular owner, in a way tliat can be used to identify the owner. 
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5 nie value of the stega-cipher is that it provides a way to watermark the content in a way 
that changes it slightly, but does not impact human perception significantly. And, 
furthermore, that it is made difficult to defeat since one must know exactly where the 
information resides to extract it for analysis and use in forgery attempts, or to remove it 
without overly degrading the signal. And, to try to forge copyright information one must 
10 first be able to analyze the enaypted copyright information, and in order to do that, one 
must be able to fmd it, which requires masks . 
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n. Example Enibodi^nnt of General Processing 



Digital audio data is rq)resented by a series of samples in 1 dimension. 
{Sp S^, S3... Sjj} 

This series is also refeired to as a sample stream. The sample stream approximates an 
analog waveform of soimd amplitude over time. Each sample rq>resents an estimate of the 
wave amplimde at the instant of time the sample is recorded. For monaural audio, fliere is 
one such sample stream. Stereo audio is comprised of two sample streams, one 
representing the right channel, and the other representing the left Each stream is used to 
drive a corresponding speaker to reproduce flie stereo sound. 

What is refered to as CD qualitj' audio is characterized by 16 bit (2 byte) stereo samples, 
recorded at 44. 1 Khz, or 44.100 samples per second in each channel. The dynamic range of 
sound reproduction is directly proportional to the number of bits per sample. Some lower 
quaUtj' recordings are done at 8 bits. A CD audio recording can be stored using any 
scheme for containing the 2 sample streams in their entirety'. When these streams are 
played back at the same frequency they were recorded at, the sound recorded is reproduced 
to a high degree of accuracy. 

Tlie sample stream is processed in order from fu st sample to last. For the purpose of the 
invention disclosed, tlie stream is separated into sample windows, each of which has a 
fixed number of consecutive samples from tlie stieam. and where windows do not overlap 
in the sample stieam. Windows may be contiguous in the sample stream. In this 
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discussion assume each window contains 128 samples, and that windows aie contiguous. 
So, the windows within the stream look like 

{ [Sj, S^, S3„.Sj2J, [Si2*S^3(,,Sj3j...S^3j,.„[S^_j28. .S J ) 

where [...] denotes each window and any odd samples at the end of the stream which do 
not completely fill a window can be ignored, and simply passed through the system 
unmodified. 

These windows will be used as input for the discrete Fast Fourier Transfomi (and its 
inverse) operation. 

Briefly, Fourier Transform methods are based on the princq)le that a complex wavefomi, 
expressed as amplitude over time and represented by a sample stream, is really the smn of 
a number of simple wavefomis, each of which oscillate at different frequencies. 

By complex, it is meant that the value of the next sample is not easily predicted from the 
values of the last N samples or tlie time of the sample. By simple it is meant that the value 
of the sample is easily predictable from the values of the last N samples and/or the time of 
tlie sample. 

The sum of multiple simple waves is equivalent to the complex wave. The discrete FFT 
and its inverse simply translate a limited amount of data from one side of tfiis equivalence 
to tlie otlier, between the complex waveform and tlie sum of simple waves. Tlie discrete 
FFT can be used to translate a series of samples representing ampUtude over time (the 
complex wave, representing a digital audio recording) into the same number of samples 
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5 representing total specti^al energy in a given range of frequencies ( tlie simple wave 

components) at a particular instant of time. This instant is the time in the middle of the 
original amplitude/time samples. The inverse discrete FFT translates tlie data in the other 
direction, producing the complex waveform, from its simpler parts. 

10 Each 128 sample window will be used as an input to the discrete FFT, resulting in 128 
bins representing each of 128 frequency bands, ranging from OHz to 22Khz (the Nyquist 
frequency, or 1/2 the sampling rate). 

Infomiation can be encoded into the audio signal in the frequ^icy domain or in tlie time 
15 domain. In the latter case, no FFT or inverse FFT is necessary. However, encoding in tlie 
frequency domain is recommended, since its effects are scattered over the resultant time 
domain samples, and not easily predicted. In addition, frequency domain encoding makes 
it more likely that randomization will result in noticeable artifacts in the resultant signal, and 
therefore makes the stega-cipher more defensible against such attacks. It is in the frequency 
20 domain that additional infomiation will be aicoded into the audio signal for tlie purpose of 
this discussion. Each frequency band in a given time slice can potentially be used to store a 
small portion of some additional information to be added to the signal. Since tfiese are 
discrete estimates, there is some room for error which will not significantly effect tlie 
perceived quahty of the signal, reproduced after modification, by the inverse FFT 
25 operation. In effect, intentional changes, which caimot be distinguished from random 
variations are introduced in tlie frequency domain, for the purpose of storing additional 
information in the sample stream. These changes are minimized so as not to adveisely 
affect tlie perceived quahty of the reproduced audio signal, after it has been encoded witli 
additional infonuation in the mamiei* described below. In addition, tlie location of each of 
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these changes is made virtually impossible to predict, an innovation which distinguishes 
this scheme from simple steganographic techniques. 

Note tliat tliis process differs from the Nagata, et al. patents » 4,979,210 and 5,073,925, 
which encode information by modulating an audio signal in amplitude/time domain. It also 
differs in that the modulations introduced in the Nagata process (which are at verj^ low 
amplitude and frequency relative to the carrier wave as to remain inaudible) carrj' onfy 
copy/ don't copy information, which is easily found and circumvented by one skilled in the 
art Also, theie is no limitation in the stega-cq>her process as to what type of information 
can be encoded into the signal, and there is more information storage capacity, since the 
encoding process is not boimd by any particular frequency of modulation but ratha: by the 
numbei- of samples available. The granularity of encoding in the stega-cipher is determined 
by the sample window size, with potentially I bit of space per sample or 128 bits per 
window (a secure implementation will halve this to 64 bits). In Nagata, et al. the granularity^ 
o^ encoding is fixed by the amplitude and frequency modulation limits required to maintain 
inaudibility. Tliese limits are relatively low, and therefore make it impractical to encode 
more than simple copy/ don't copy infomiation using the Nagata process. 




5 III- Example Embodiment of Encoding and Decoding 

A modification to standard steganograpliic technique is applied in the frequency domain 
described above, in order to encode additional infonnation into the audio signal, 

10 In a scheme adapted from cryptographic techniques , 2 keys are used in the actual encode 
and decode process. For the purposes of ttiis invention the keys are refered to as masks. 
One mask, the primary, is appUed to the frequency axis of FFT results, the other mask is 
apphed to the lime axis (this will be called the convolution mask). The number of bits 
comprising the primary mask are equal to the sample window size in samples (or fee 

15 numbei' of frequency bands computed by the FFT process), 128 in this discussion. The 
number of bits in the convolution mask are entirely arbitrary. This implementation will 
assume a time mask of 1024 bits. Generally ttie larger tiie key, the more difficult it is to 
guess. 

20 Prior to encoding, the primary and convolution masks described above are generated by a 
cryptographically secure random generation process. It is possible to use a block cq^her like 
DES in combination witli a sufficiently pseudo-random seed value to emulate a 
cryptographically secure random bit generator. These keys will be saved along with 
infonnation matching tliem to tlie sample stream in question in a database for use in 

25 decoding, should that step become necessarj'. 

Prior to encoding, some additional infonnation to be encoded into tlie signal is prepaied 
and made available to the encoder, in a bit addressable maimer (so tliat it may be read one 
bit at a time). If the size of the sample stream is known and the efficiency characteristics of 
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5 the stega-cipher implementation are taken into account, a known limit may be imposed on 
the amount of this additional infonnation. 

Tlie encoder captures one sample window at a time from tlie sample stream, in sequential, 
contiguous order. The CTicoder tracks the sequential number of each window it acquires. 
10 The first window is 0. When ttie mmiber of windows processed reaches the number of bits 
in the window mask, minus one, the next value of the window counter will be reset to 0. 

This counter is the convolution index or phase. In tlie cuirent implemetation it is used as a 
simple index into flie convolution bitmask. In anticipated developments it will be used to 
1 5 perfomi convolution operations on ttie convolution mask to detemiine which bit to use. 

For instance the mask might by rotated by a number corresponding to the phase, in bits to 
the left and XORed with the primary mask to produce a new mask, which is then indexed 
by the phase. There are many possibilities for convolution. 

20 Tlie encoder computes the discrete FFT of the sample window. 

Starting with the lowest frequency band, the encoder proceeds through each band to the 
highest, visiting each of the 128 frequency bands in order. At each band value, the encoder 
takes the bit of the primary mask corresponding to the frequency band in question, the bit 
25 of the convolution mask corresponding to the window in question, and passes fliese values 
into a boolean function. This function is designed so that it has a near perfectly random 
output distribution. It will return true for ^proxmiately 50% of its input permutations, and 
false for flie other 50%. The value returned for a given set of inputs is fixed, however, so 
that it will always return the same value gi\'en the same set of inputs. 
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5 

If the function returns true, the current frequency band in the current window is used in the 
encoding process, and represents a vaUd piece of the additional information encoded in tlie 
signal. If the function returns false, this cell, as the frequency band in a given window is 
caUed, is ignored in the process. In this manner it is made extremely difficult to exti-act the 

1 0 encoded information from the signal without the use of tlie exact masks used in the 
encoding process. This is one place in which the stega-cq)her process departs from 
traditional steganographic implementations, which offer a trivial decode opportunity if one 
knows the information is presoit While this increases the information storage capacity' of 
ttie carrier signal, it makes decoding trivial, and further degrades the signal. Note that it is 

1 5 possible and desireable to modify the boolean cell flag function so that it returns true < 
50% of the time. In general, the fewer cells acmally used in the encode, the more difficult 
they will be to find and the less degradation of content will be caused, provided the function 
is designed correctly. There is an obvious tradeoff in storage capacity for this increased 
security and quality. 

20 

Tlie encoder proceeds in this manner until a complete copy of the additional information 
has been encoded in the carrier signal. It will be desireable to have ttie encoder encode 
multiple copies of the additional information continuously over the duration of tiie carrier 
signal, so that a complete instance of this information may be recovered from a smaller 
25 segment of a larger signal which has been split into discontinuous pieces or otherwise 

edited. It is tlierefore desireable to minimize tlie size of the information to be encoded using 
botli compact design and pre-encoding compression, thus maximizing redundant encoding, 
and recoverabilit)' from smallea" segments. In a practical implementation of tliis system it is 
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5 likely the information will be first compressed by a known method, and then enciypted 
using public-key techniques, before being encoded into the canier signal. 

Hie encoder will also prepare ttie package of additional information so tliat it contains an 
easily recognizable start of message delimeter, which can be unique to each encoding and 

10 stored along with the keys, to serve as a synchronization signal to a decoder. The detection 
of tliis delimeter in a decoding window signifies that the decoder can be reasonabty suie it 
is aligned to the sample stream correctly and can proceed in a methodic window by 
window manner. These delimeters will require a number of bits which minimizes the 
probability that this bit sequence is not reproduced in a random occurraice, causing an 

1 5 accidental misaligment of the decoder. A minimum of 256 bits is recommended. In the 
ciin ent implementation 1024 bits representing a start of message delimeter are used. If 
each sample is random, then each bit has a 50% probably of matching the delimeter and the 
conditional probability of a random match would be 1/2^^''*. In practice, the samples are 
probably somewhat less than random, increasing the probability of a match somewhat. 

20 

The decode process uses the same masks in tlie same manner, only in this case the 
infomiation is extracted one bit at a time from the carrier signal. 

Tlie decoder is assumed to have access to the proper masks used to encode ttie infonnation 
2 5 origmally . These masks might be present in a database, which can be indexed by a value, 
or values computed from tlie original content, in a manner insensitive to tlie modifications 
to the content caused by tlie stega-cipher process. So, given an arbitrary piece of content, a 
decoder might first process the content to generate certain key values, and tlien retrieve the 
decode masks associated with the matching key values from tlie database. In the case 
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where multiple matches occur, or none are found, it is conceivable that all mask sets in tlie 
database could be tried sequentially until a valid decode is achieved, or not, indicating no 
information is present 

In the application of this process, it is anticipated that encoding operations may be done on 
a given piece of contmt up to 3 times, each adding new information and using new masks, 
ovei- a sub-segment of tiie content, and that decode operations will be done infrequently. It 
is anticipated that should it become necessary to do a search of a large number of masks to 
find a valid decode, that this process can be optimized using a guessing technique based on 
close key matching, and ttiat it is not a time critical application, so it will be feasible to test 
lai'ge numbers of potential masks for validity on a given piece of content, even if such a 
process takes days or weeks on powerful computers to do a comprehensive search of 
known mask sets. 

'Hie decode process is slightly different in the following respect Whereas the encoding 
process can start at any arbitrary point in the sample stream, the decode process does not 
know where the encode process began (the exact offset in samples to the start of the first 
window). Even though the encode process, by convention, starts with sample 0, there is no 
guarantee tliat tlie sample stream has not been edited since encoding, leaving a partial 
window at the start of the sample stream, and thus requiring the decoder to find the first 
complete window to start the decode. Therefore, the decode process will start at the first 
sample, and shift the sample window along by 1 sample, keeping the window index at 0, 
until it can find a valid decode delimeter encoded in tlie window. At tliis point, tlie decoder 
knows it has synchronized to the encoder, and can Uien proceed to process contiguous 
windows in a more expedient manner. 
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5 

Example Calculations based on flie described implementation for adding copyright 
certificate information to CD quality digital audio: 

In a stream of samples, every 128 samples will contain, on average 64 bits of certificate 
related information. Digital audio is composed of 16 bit samples, at 44.1 Khz, or 44,100 
samples per second. Stereo audio provides 2 streams of information at tliis rate, left and 
right, or 88,200 samples per second. That yields approximately 689 contiguous sample 
windows (of 128 samples) per second in which to encode information. Assume a song is 
4 minutes long, or 240 seconds. This yields 240 * 689 = 165,360 windows, which on 
average (50% utilization) contain 64 bits (8 bytes) each of certificate information. This in 
turns gives ^proximately 1291Kb of information storage space per 4 minute stereo song 
(1.2 MB). There is ample room for redundant encoding of infomiation continuously over 
the l^gth of the content Encoding 8 bytes for every 256 bytes represents 3.1% of the 
signal infomiation. Assuming that a copyright certificate requires at most approximately 
2048 bytes (2K), we can encode the same certificate in 645 distinct locations within the 
recording, or approximately every 37/lOOths of a second. 

Now to account for delimeters and synchronization information. Assuming a sync marker 
of 1024 bits to avoid random matches, tlien we could preftx each 2K ceitificate block with 
25 this 1024 bit marker. It takes 256 windows to store 2K, and under fliis proposed scheme, 
tlie first 16 windows are reserved for the sync marker. A decoder could seaich for this 
marker by progressively matching each of the first 16 windows (64 bits at a time) against 
tlie conesponding portion of the sync markei*. The decoder could reset tlie match advancuig 
through the sample stream, as soon as one window did not confomi to the sync markei', 
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and proceed in this manner until it matches 16 consecutive windows to the marker, at 



which point it is synchronized. 

Under this scheme, 240 windows, or 1.92K remain remain for storing certificate 
information, which is notimreasonable. 

10 

IV. Possfble Problems, Attacks and Subsequoit Drfenses 

A, Randomization 

The attacker simply randomizes the least significant bits of each data point in the transform 
15 buffer, obUterating the synchronization signal and the watermark. While this attack can 
remove the watemiark, in the context in which stega-cipher is to be used, the problem of 
piracy is kept to a minimimi at least equal to that afforded by traditional media, since the 
system will not allow an unwatermarked piece of content to be traded for profit and 
w^temiarks camiot be forged without ttie proper keys, which are computationally difficult 
20 10 obtain by brute-force or crytanalysis. In addition, if the encoding is managed in such a 
way as to maximize the level of changes to the sample stream to be just at the threshold 
below human perception, and the scheme is implemented to anticq?ate randomization 
attempts, it is possible to force the randomization level to exceed the level that can be 
peiceived and create destructive artifacts in the signal, in much the same manner as a VHS 
2 5 cassette can be manufactured at a minimal signal level, so that a single copy results in 
unwatchable static. 

B, LoM' Bit-Depth Bitmaps (black & white images) 
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5 These bitmaps would be too sensitive to the steganization process, resulting in unacceptable 
signal degradation, and so are not good candidates for the stega-c^her process. The 
problem may be circumvented by inflating bit-depth, although this is an inefficient use of 
space and bandwidth. 

10 C. Non-Integer Transforms 

The FFT is used to gaierate spectral energy information for a given audio signal. This 
information is not usually in integer format Computers use methods of approximation in 
tliese cases to represent the real numbers (whole numbers plus fractional amoimts). 
Depending on tiie exact value of the nimiber to be represented slight errors, produced by 

1 5 rounding off the nearest real nxmiba: that can be completely specified by the computer 
occur. This will produce some randomization in the least significant bit or bits. In other 
words, the same operation on the same sample window might yield slightly differait 
transform values each time. It is possible to circumvent this problem using a modification 
to the simple LSB steganographic technique described later. Instead of looking at tlie LSB, 

20 it might be possible to use an energy quantization technique in place of the LSB method. 

Some variant of rounding the spectral energy values up or down, with a granularity' greater 
than the rounding error should work, without significantly degrading the output samples. 

V. A Method and Protocol For Using the Stega-Cipher 

25 

Tlie apparatus described in the claims below operates on a window by window basis over 
the sample stream. It has no knowlege of the nature of the specific message to be encoded. 
It merely indexes into a bit stream, and encodes as many of tfiose bits as possible into a 
given sample window, using a map determined by the given masks. 
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Tlie value of encoding information into a single window in the sample stream using such 
an apparatus may not be inherently apparent until one examines the manner in which such 
information will be used. The protocol discussed in this section details how messages 
which exceed the encoding capacity of a single sample window (128 samples) may be 
10 assembled from smaller pieces encoded in the individual windows and used to defend 
copyrights in an online situation. 

An average of 64 bits can be encoded into each window, which equals only 8 bytes. 
Messages larger than 8 bytes can be encoded by simply dividing tlie messages up and 
1 5 encoding small portions into a string of consecutive windows in the sample stream. Since 
the keys determine exactly how many bits will be encoded per window, and an element of 
randomness is desirable, as opposed to perfect predictabihty, one cannot be certain exactly 
how many bits are aicoded into each window. 

20 The start of each message is marked by a special start of message delimeter, which, as 

discussed above is 1024 bits, or 128 bytes. Therefore, if precisely 8 bytes are ^coded per 
window, tlie first 16 windows of any useable message in the system described hare are 
reser\'ed for tiie start of message delimeter. For the encoder, this scheme presents little 
challenge. It simply designates the first sample window in tlie stream to be window 0, and 

2 5 proceeds to encode the message delimeter, bit-by-bit into each consecutive window. As 

soon as it has processed the last bit of the SOM delimeter it continues by encoding 32 bits 
representing the size, in bytes of tlie complete message to follow. Once tlie 32nd and final 
bit of the size is encoded, the message itself is encoded into each consecutive wmdow, one 
bit at a time. Some windows may contain more encoded bits then others, as dictated by tlie 
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masks. As the encoder processes each window in the content it increments its window 
counter. It uses this counter to index into the window mask. If the nimiber of windows 
required to encode a complete message is greater than flie size of this mask, 256 bits in this 
case, or 256 windows, then it simply resets the counter after window 255, and so on, until 
a complete message is encoded. It can then start over, or start on a new* message. 

The decoder has a bigger challenge to face. The decoder is given a set of masks, just like 
encoder. Unlike the encoder, the decoder cannot be sure that the first series of 128 samples 
it recieves are the window 0 start of message, encoded by the decoder. The sample stream 
originally produced by an encoder may have been editted by clipping its ends randomly or 
splicing pieces together. In ttiatcase, the particular copy of the message tiiat was cl^jped is 
unrecoverable. The decoder has the start of message delimeta: used to encode the message 
that the decoder is looking for. In the inital state, the decoder assimies the first window it 
gets is window 0. It then decodes the prop^ number of bits dictated by the masks it was 
given. It compares these bits to the corresponding bits of tiie start of message delimeter. If 
they match, the decoder assumes it is still aUgned, increments the window counts and 
continues. If the bits do not match, the decoder knows it is not aligned. In this case, it shifts 
one more sample onto tiie end of the sample buffer, discarding the first sample, and starts 
over. The window counter is set to 0. The decoder searches one sample at a time for an 
alignment lock. The decoder proceeds in this manner until it has decoded a complete match 
to the start of of message delimeter or it exhausts the sample stream without decoding a 
message. If flie decode can match completely tlie start of message delimeter bit sequence, 
it switches into aligned mode. Tlie decoder will now advance tlu*ough tlie sample sUeam a 
full window at a time (128 samples). It proceeds until it has the 32 bits specifying the 
message size. This generally won't occupy more than 1 complete window. When the 
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decoder has locked onto the start of message delinieter and decoded the message size, it 
can now proceed to decode as many consecutive additional windows as necessary until it 
has decoded a complete messsage. Once it has decoded a complete message, the state of 
tlie decoder can be reset to un-synchronized and the entire process can be repeated starting 
with die next 128 sample window. In this manner it is not absolutely necessar}' that 
encoding windows be contiguous in the sample stream. The decoder is enable of handling 
random intervals between tiie end of one message and ttie start of anotlier. 

It is important to note that the circuit for encoding and decoding a sample window does not 
need to be aware of the nature of the message, or of any structure beyond the start of 
message delimeter and message size. It only needs to consider a single sample window, its 
own state (whether the decoder is misaligned, synchronizing, or synchronized) and what 
bits to aicode/decode. 

KJiven that the stega-cipher apparatus allows for the encoding and decoding of arbitrary 
messages in this maimer, how can it be used to protect copyrights? 

Hie most important aspect of the stega-cipher in this respect is that fact that it makes the 
message integral with the content, and difficult to remove. So it cannot be eliminated 
simply by removing certain information prepended or upended to the sample stream 
itself. In fact removing an arbitrary chunk of samples will not generaUy defeat the stega- 
ciphei' either. 

Given that some information can be thus integrated with the content itself, tiie question is 
then how best to take advantage of this arrangement in order to protect copyrights. 



28 




5 

Tlie following protocol detaUs how the stega-cipher will be exploited to protect copyrights 
in the digital domain. 

In a transaction involving the transfer of digitized content, there are at least 3 functions 
10 involved: 

The Authority is a trusted arbitrator between the two other functions listed below, 
representing parties who actually engage in the transfa- of the content The Authority 
maintains a database containing information on the particular piece of content itself and 
1 5 who the two parties engaged in transfering the contait are. The Authority can perform 
stega-cipher encoding and decoding on content 

The Publisher, or online distributor is the entity which is sliding the copyrighted contmt to 
another party. The Publisher can perform stega-cipher encoding and decoding on content 

20 

The Consumer is the person or entit>' receiving the copyrighted content, generally in 
exchange for some consideration such as money. The consumer caimot generally perfomi 
stega-cipher encoding or decoding on content 

25 Each of these parties can participate in a message exchange protocol using well known 
public-key cryptographic techniques. For instance, a system licensing RSA public key 
algorithms might be used for signed and encrypted message exchange. This means tliat 
each paity maintains a public key / private key pair, and that tiie public keys of each party 
are freely available to any otiier partj'. Generally, the Authority communicates via electronic 
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5 links directly only to the Publisher and the Consumer communicates directly only with tlie 
publisher. 

Below is an example of how the protocol operates from the time a piece of content enters 
an electronic distribution system to the time it is delivered to a Consumer. 

10 

A copyright holder (an independent artist, music publisher, movie studio, etc,) wishes to 
retail a particular title online. For instance. Sire Records Company might wish to distribute 
the latest single from Seal, one of their musical artists, online. Sire delivers a master copy 
of tliis single, "Prayer for the Dying", to the Authority, Ethical Inc. Ethical converts the 

15 title into a format suitable for electronic distribution. This may involve digitizing an analog 
recording. The title has now become content in the cont^t of this online distribution 
system. The title is not yet available to anyone except Ethical Inc., and has not yet been 
encoded with the stega-ciphCT watamark. Ethical generates a Title Identification and 
Authentication (TIA) certificate. Tlie catificate could be in any fomiat. In this example it is 

20 a short text file, readable with a small word-processing program, which contains 
infomiation identifying 

the title 
tlie artist 

25 the copyright holder 

tlie body to which royalties should be paid 

general temis for publishers* distribution 

any other information helpful in identifying fliis content 
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Ethical then signs the TIA wifli its own private key, and encrypts the TIA certificate plus its 
signature witti its own public key. Thus, the Ethical can decrypt the HA certificate at a later 
tinie and know that it generated the message and tliat the contents of the message have not 
been changed since generation. 

10 

Sire Records, which ultimately controls distribution of flie content communicates to the 
Ethical a specific online PublishCT that is to have the right of distribution of this content 
For instance, Joe's Online Emporium. The Authority, Ethical Inc. can transmit a short 
agreOTient, theDistribution Agreanentto the Publish^-, Joe*s Online Emporium which 
15 lists 

the content title 
tlie publisher's identification 
\ the terms of distribution 
20 any consideration paid for the right to distribute the content 

a brief statement of agreement with all terms listed above 

The Publisher receives this agreement, and signs it using its private key. Thus, any party 
with access to the Joe's Online Emporium's public key could verify that the Joe's signed 
25 the agreement, and that the agreement has not been changed since Joe's signed it The 
Publisher transmits the signed Distribution Agreemmt to tfie Authority-, Ethical Inc. 

Etliical Inc. now combines tlie signed TIA certificate and the Distribution Agreement into a 
single message, and signs the entire message using its private key. Ethical has now created 
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5 a Publisher Identification message to go into its own stega-cipher channel in tlie content. 
Ethical Inc. now generates new stega-cq)her masks and encodes this message into a copy 
of the content using a stega-cipher encoder. The Authority saves the masks as a Receipt in 
a database, along with information on the details of the transfer, including the title, aitist 
and publisher. 

10 

Ethical tlien transfers this watemiarked copy to the Joe's Online Emporium, tlie Publisher. 
Well known encryption methods could be used to protect the transfer between the 
Authority and the Publisher. The Authority may now destroy its copy, which the Publisher 
has received. The Publisher, Joe's Online Emporium now assumes responsibility'^ for any 
15 copies made to its version of the content, which is a Publisher Master copy. 

Finally, the Consimier, John Q. Public wishes to purchase a copy of the content from Joe's 
Online Emporium. Joe's Emporium sends the John Q. Public a short agreement via an 
el^tronic communication link, similar to Publisher's Distribution Agreement, only this is 
20 a Purchas e A greem ent, which lis ts 

the content title 
consumer identification 
the terms of distribution 
2 5 the consideration pas for the content 

a brief statement of agreement with the terms above 

John Q. Pubhc signs this agreement with his private key and returns it to the Joe's Online 
Emporium. The Publisher, Joe's prepares to encode its own stega-cq>her watemiark onto a 
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5 copy of the content by generating a set of masks for tlie algorithm. Joe's Online 

Emporium then stores these masks (a receipt) in its own database, indexed by title and 
consumer. Joe's Online Emporium signs the agreement recei\'ed from John Q. Public 
with the Emporium's own private key, and forwards it to the Authority'. Efliical Inc., along 
with a copy of the masks. It is important to note that this communication should be done 
10 over a secured channel. The Authority voifies the PubUsher and Consumer information 
and adds its own signature to the end of the message, approving the transaction, creating a 
Contract of Sale. The Authority adds the Publisher's receipt (mask set) to its database. 
inde.xed by the title, the publisher, and the consumer identification. The Authority signs the 
Contract of Sale by encrypting it with their private key. So anyone with the Authorit\''s 
1 5 pubhc key (any Publisher) could decrypt the Contract of Sale and verif>' it, once it was 

extracted from fee content The Publisher then transmits the signed Contract of Sale back to 
the Pubhsher, who uses a stega-cipher device to imprint this Contract as its own watermark 
over the content The PubUsho- then transmits the newly watamarked copy to the 
(Consumer, who is accepting responsibUity for it The Publisher destroys tlieir version of 
20 the consumer's copy. 

If this procedure is followed for all content distribution within such an online system then it 
should be possible for the Authority to identify the owner of a piece of content which 
appears to be unautiiorized. The Autiiority could simply try its database of stega-cipher 
2 5 keys to decode the watennark in the content in question. For instance, if a copy of Seal's 
latest single originaUy distributed witii stega-cipher watermarks showed up on an Internet 
ftp site the Autliority should be able to extract a TIA Certificate and Distribution 
Agreement or a Conduct of Sale identifying tlie responsible party. If a Publisher sold tiiis 
particular copy to a Consumer, ttiat particular publisher should be able to extract a Conti^t 
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5 of Sale, which places responsibUity with the Consumer. This is not a time critical 
application, so even if it takes days or weeks, it is stiU worthwhile. 

In a modification to the protocol discussed above, each PubUsher might act as its own 
Authorit>'. However, in the context of online services, this could open avenues of fraud 
10 committed by the coUusion of certain Publishers and Consumers. Using an Authoritj', or 
one of sevo^I avaUable Authorities to keep records of Publisher-Consumer transactions 
and verify their details decreases the likelihood of such events. 

It should also be obvious that a similar watennaiking system could be used by an 
1 5 individual entity to watermark its own content for its own purposes, wether online or in 
physical media. For instance, a CD manufacture could incorporate unique stega-cipher 
watermarks into specific batches of its compact discs to indentify the source of a pirate 
ring, or to identify unauthorized digital copies made from its discs. This is possible because 
^he stega-cqjher encoding works with the existing formats of digital samples and does not 
20 add any new structures to the sampe data that cannot be handled on electronic or 
mechanical systems which predate the stega-cqjher. 



VI. Increasing Confidoice In the Stega-Cipher 

25 The addition of a special pre-encoding process can make stega-cipher certificates even 
more secure and undeniable. Hash values may be incorporated which match exactly the 
content containing the watermark to tlie message in die watermark iteelf. This allows us a 
verification that the watermark decoded was encoded by whomever signed it into tliis 
precise location in this specific content 
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Suppose one wants to use a 256 bit (32 byte) hash value which is cakulated with a secure 
one-way hash function over each sample in each sample window that will contain the 
message. The hash starts with a seed value, and each sample that would be processed by 
the encoder when encoding the message is incorporated into the hash as it is processed. 
10 The result is a 256 bit number one can be highly confident is unique, or sufficiently rare to 
make intentionally duplicating it with another series of samples difficult 

It is important that the hash function be insensitive to to any changes in the samples 
induced by the stega-cipher itself For instance, one might ignore the least significant bit of 
1 5 each sample when computing tlie hash function, if the stega-cipher was implemented using 
a least significant bit aicode mode. 

Based on the size of the non-hash message, one knows the hash-inclusive message 
Requires 32 more bytes of space. One can now cafculate the size of a signed encrypted copy 
20 of this message by signing and encrypting exactiy as many random bytes as are in tiie 
message, and measuring tiie size of the output in bytes. One now knows the size of the 
message to be encoded. One can pre-process the sample stream as foUows. 

Proceed throught ttie stega-cipher encode loop as described in the claims. Instead of 
2 5 encoding, however, calculate hash values for each window series which will contain the 
message, as each sample is processed. At the end of each instance of "encoding" take tiie 
resultant hash value and use it to aeate a unique copy of ttie message which includes the 
hash value particular to tiie series of sample windows tiiat will be used to encode tiie 
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5 message. Sign and encrypt this copy of the message. ai.d save it for encoding in a.e san.e 
place in the sample stream. 

A memory effHcient version of this scheme could keep on hand the un-hashed message, 
and as it creates each ne>v copy, back up in the sanaple stream to the first window in the 
10 series and actually encode each message, disposing of it afterwards. 

The important result is evident on decoding. The decoding party can calculate the same 
hash used to encode the message for themselves, but on the encoded samples. If the value 
calculated by the decoding party does not match the value contained in fte signed message. 
1 5 the decoder is alerted to the fact that this wateimark was transplanted from somewhere 
else. TTiis is possible only with a hash function which ignores the changes made by the 
stega-cipher after the hash in the watermark was generated. 

TiMs scheme makes it impossfl>le to transplant watennarks. ev«i with the keys to the stega- 
20 cipher. 
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Claims 



What Is Claimed Is 



1) A method, caUed stega-cipher, comprising the step of using random keys in 
combination with steganographic technique to encode additional infonnation into digitized 
samples in a manner that does not significantly degrade the signal generated from the 
sample stream and makes it impossible to extract the infonnation without the keys 
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An electronic, digital apparatus for stega-cipher encoding or decoding a message, 
represented as a series of data bits, into or out of a series of digitized samples comprising: 

V) a high speed sample buffer, for holding and quickly accessing and transforming 128 
digitized samples 

b) a digital signal processor or microprocessor powerful enoug^o perform fast fourier 
transforms 

c) a high speed memory, in the form of RAM or SR.\M to contain information, 
representing 

1) 128 bit primary mask 

2) 256 bitconvolutional mask 
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5 3)1024 bit start of message delimetei' 

4) 256 bit mask calculation buffer 

5) 2 Kilobyte message buffer 

6) an integer rq>resenting a message bit index 

7) an integer respresenting message size 

10 8)an integer representing an index into the primary mask 

9) an integer representing an index into the convolutional mask 

10) an integer representing the state of the decode process 

11a boolean truth table representing the stega-cipher map function 
12) a boolean flag indicating a complete message has been decoded, or encoded 
15 13) an integer representing the number of samples to read into the sample 

buffer 

14) a boolean flag indicating the size of a message has been decoded 

d) a means to acquire digital samples from an attached digital circuit or analog input 

20 e) a means to output digital samples to an attached digital circuit or analog output 

f) a means to recieve the values of [cl] - [c5] and [cll] from an attached digital circuit 

e) a means to output tlie message stored in [c5] as the result of a decode process and ttie 
2 5 value of [c 1 1 ] to an attached digital circuit 

f) at least one data bus to transfer infomiation from • 

[d] to [a] 
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[a] to [b] 

[b] to [a] 
[a] to [e] 
[f] to [c] 

[c] to [e] 



a clock signal to drive [b] and control the operation of the circuit 



3) 



a meUiod for utUizing the apparatus of claim 2 to encode stega-cipher infonnation into 
sample stream of data, wherein said method is comprised of the foUowing steps 



I. Initialization 

Generate a mask set to be used for encoding 

the set includes a random 128 bit primary mask 

a random 256 bit convolution mask 

a random 1024 bit start of message delimeter 

Obtain a message to be encoded 

compress and encrypt the message if desired 
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( 

Generate a message bit stream to be encoded such that the stream is comprised of 
1024 bits of start of message delimeter 
32 bits representing the number of message bytes to follow 
{ the bytes of tiie message, optionally compressed and/or encr>'pted } 
in that order 

load the message bit stream , tiie stega-cipher map trutli table, the prmian' mask, 
the convolution mask and the start of message delimeto- into [c] 



ttie processor sets the primary and convolution mask indexes to 0, and sets tlie 
1 5 message bit index to 0, and sets the message size integer equal to the total number 

of bits in the message bit stream 

the processor sets the message aicoded flag to false 

20 n. tlie processor reads a window of 128 samples from the sample input means and stores 
them sequentially in the sample buffer 
if this step fails, the process ends 

m. The processor sets the primary mask index to 0 and uses it to loop ttirough the sample 
2 5 buffer from tlie first sample to the last sample, incrementing the index each time, visiting 
each sample. For each sample position, the processor computes the value of tlie mapping 
function, which is either true or false. To do tliis, tlie processor uses tlie bit of tlie primary 
mask representing the current sample (0-127) and the bit of the convolution mask indicated 
by tlie convolution index (0-255) to calculate an offset into the truth table representing tlie 
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mapping function. The processor then obtains the bit value stored there. If the value true 
(1), the processor encodes the bit of the message indicated by the message bit index into 
the cuxrent sample and incremaits the message bit index. If at any tinie during tliis loop the 
message bit index equals the total message bits, then the processor performs step I. sets the 
message encoded flag and exits the loop. 

IV. The processor outputs the samples in the sample buffer via the sample output means. 
If the message encoded flag is set, the processor turns the flag off, and jumps back to step 

n. 

V. The processor increments the convohition index. If the convolution index equals the 
length of the convolution mask in bits (256), th«i the processor sets the convolution index 
to 0. Finalfy, the processor jumps back to step E. 

4) a method for utilizing the apparatus of claim 2 to encode stega-cipher information into a 
sample stream of data, wherein said method is comprised of the following steps 



I. Initialization 

Generate a mask set to be used for encoding 

the set includes a random 128 bit primary mask 

a random 256 bit convolution mask 

a random 1024 bit start of message delimeter 
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Obtain a message to be encoded 

compress and encrypt the message if desired 

Generate a message bit stream to be encoded such that the stream is comprised of 
1024 bits of start of message delimeter 
32 bits representing tlie number of message bytes to follow 
{ the bytes of ttie message, optionally compressed and/or encrypted ) 
in that order 

load the message bit stream , the stega-cipher map truth table, the primary mask, 
the convolution mask and the start of message delimeter into [c] 

the processor sets the primary and convolution mask indexes to 0, and sets the 
^ message bit index to 0, and sets tlie message size integer equal to tlie total number 
of bits in the message bit stream 

11. the processor reads a window of 128 samples from the sample input means and stores 
them sequentially in the sample buffei* 
if this step fails, the process ends 

IIA. The processor computes the Fast Fourier Transfomi of tlie samples in the message 
buffer, replacing them with spectral sample values. 
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5 in. The processor uses the primary mask index to loop through the sample buffer from flie 
first sample to the last sample, visiting each one. For each sample position, the processor 
computes the value of the mapping function, which is either true or false. To do this, tlie 
processor uses the bit of the primary mask representing the current sample (0-127) and tlie 
bit of the convolution mask indicated by the convolution phase (0-255) to calculate an 
10 offset into the truth table representing the mapping function. The processor then obtains the 
bit value stored there. If ttie value is true, ttie processor encodes the bit of the message 
indicated by the message bit index into the current sample and increments the message bit 
index. If at any time during ttiis loop the message bit index equals flie total message bits, 
then the processor performs step I, sets the message encoded flag and exits the loop. 

15 

IV. The processor outputs the samples in the sample buffer via the sample output means 

IVA. Tlie processor computes the inverse Fast Fourier Transform of the spectral sample 
values in the sample buffer, converting them back to ampUtude samples. If the message 
20 encoded flag is on, the processor turns it off and jumps to step 11. 

V. The processor increments the convolution index. If the convolution index equals the 
length of the convolution mask in bits (256), theh the processor sets ttie convolution index 
to 0. Finally, ttie processor jimips back to step II. 

25 

5) Tlie method of claims 3 or 4 where the encoding of the message bit into the sample in 
step in consists of altering ttie least significant bit of flie sample to match tlie message bit 
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6) The method of claims 3 or 4 where the encoding of the message bit into tlie sample in 
step rn consists of altering the sample value so that it falls within a specified range of 
values relative to its orginial value. For instance, if the message bit is 0, the sample value 
must have a least significant tens digit of 0-4, and if tlie message bit is 1, the sample value 
must have a least significant tens digit of 5-9. If the message bit is one, and the original 
sample value is 1003, then flie sample value could be changed to [995-999] or [1005-1009] 
in order to signifs' a 1 message bit If tlie message bit were 0 under this scheme, ttie sample 
could assume any value [1000-1004] or could be left as is. 

7) 

a method for utilizing ttie apparatus of claim 2 to decode stega-cq)her information from a 
sample stream of data, wherein said method is comprised of the following steps 

reinitialization 

Obtain a mask set to attempt decoding with 

tlie set includes a 128 bit primary mask 

a 256 bit convolution mask 

a 1024 bit start of message delinieter 

load the stega-cipher map truth table, the primary mask, the convolution mask and 
the start of message delimeter into [c] 

the processor sets the primary and convolution mask indices to 0, and sets the 
message bit index to 0, and sets the message size integer equal to 0 
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the processor sets the message decoded flag to false 

the processor sets the state of the decode process to UNSYNCHRONIZED 

10 

II. Tlie processor checks the state of the process. If it is UNSYNCHRONIZED, &e 
processor sets the number of samples to read equal to 1 and sets the convohition index to 
0. Otherwise, the processor sets the number of samples to read equal to 128 

15 EC. The processor reads the number of samples specified in [cl3] and stores th^ in tiie 
sample buffer. If this step fails the process ends, with no message decoded. 

IV. The processor sets the primary mask index to 0 and uses it to loop through the sample 
buffer from the first sample to the last sample, incrementing the index each time» visiting 

20 each sample. For each sample position, the processor computes the value of the mapping 
function, which is either true or false. To do tliis, the processor uses the bit of the primary 
mask representing the current sample (0-127) and the bit of the convolution mask indicated 
by tlie convolution phase (0-255) to cafculate an offset into the truth table representing the 
mapping function. The processor then obtains the bit value stored there. If the value is true, 

25 the processor decodes fee bit of the message indicated by the message bit index into the 
current sample, stores flie bit into the message buffer at the message bit index and 
increments the message bit index. 
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V. If the state is UNSYNCHRONIZED or SYNCHRONIZING, the processor compares 
the decoded bits in the message buffer to the start of message dehmiter. If the number of 
bits in the message buffer is less than the number of bits in the start of message delimiter 
10 and the bits match, the processor sets the state to SYNCHRONIZING. Otherwise, if the 
first 1024 bits in the message buffer match the start of message delimiter the processor 
sets the state to SYNCHRONIZED. Otherwise, the processor sets the state to 
UNSYNCHRONIZED. 

15 VI. If the state is SYNCHRONIZED and there are at least 1056 bits in the message buffer, 
the processor sets the state to SYNCH_AND_SIZE and copies bits 1024-1055 of the 
message buffer to the message size container. 

VIL If the state is SYNCH_AND_SIZE and the (number of bits in the message buffer - 
20 1056) divided by 8 is greater than or equal to the message size then a complete message is 
in the message buffer and the process ends 

VIII. The processor increments the convolution index. If the convolution index equals the 
number of.bits in the convolution mask, the processor sets the convolution index to 0. The 
25 processor jumps to step 11. 

8) 

a method for utilizing the apparatus of claim 2 to decode stega-cipher information from a 
30 sample stream of data, wherein said method is comprised of the following steps 
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L Initialization 

Obtain a mask set to attempt decoding with 

the set includes a 128 bit primary mask 

a 256 bit convolution mask 

a 1024 bit start of message delimeter 

load the stega-cipher map truth table, the primary mask, the convolution mask and 
the start of message delimeter into [c] 

the processor sets the primary and convolution mask indices to 0, and sets the 
message bit index to 0, and sets the message size intego* equal to 0 

the processor sets the message decoded flag to false 

the processor sets the state of the decode process to UNSYNCHRONIZED 



II. The processor checks the state of ttie process. If it is UNSYNCHRONIZED, the 
processor sets the number of samples to read equal to 1 and sets the convolution index to 
0. Otlierwise, the processor sets tlie number of samples to read equal to 128 

III. The processor reads tlie number of samples specified in [cl3] and stores them in tlie 
sample buffer. If this step fails the process ends, with no message decoded. 
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nia The processor computes the Fast Fourier Transform of the samples in the sample 
buffer and replaces them with frequency domain samples. 

IV, The processor sets tlie primary mask index to 0 and uses it to loop tlirough the sample 
10 buffer from the first sample to the last sample, incrementing the index each time, visiting 
each sample. For each sample position, tlie processor computes the value of the mapping 
function, which is either true or false. To do this, the processor uses the bit of the primary' 
mask representing the current sample (0-127) and the bit of the convolution mask indicated 
by the convolution phase (0-255) to cafculate an offset into the truth table representing the 
1 5 mapping function. The processor then obtains the bit value stored there. If the value is true, 
ttie processor decodes the bit of the message indicated by the message bit index into the 
current sample, stores the bit into the message buffer at the message bit index and 
increments the message bit index. 

20 V. If the state is UNSYNCHRONIZED or SYNCHRONIZING, the processor compares 
tlie decoded bits in the message buffer to the start of message delimeter. If the number of 
bits in tlie message buffer is less than ttie number of bits in the start of message delimeter 
and the bits match, the processor sets the state to SYNCHRONIZING. Otherwise, if the 
fust 1024 bits in the message buffer match the start of message delimeter the processor 

25 sets the state to SYNCHRONIZED. Otherwise, the processor sets the state to 
UNSYNCHRONIZED. 
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VI. If the state is SYNCHRONIZED and there are at least 1056 bits in the message buffer, 
the processor sets the state to SYNCH_AND.SIZE and copies bits 1024-1055 of the 
message buffa* to the message size container. 

vn. If the state is S YNCH_AND_SIZE and the (number of bits in the message buffer - 
1056 ) divided by 8 is greater flian equal to tiie message size then a complete message is in 
the message buffer and the process ends 

Vin. Tlie processor increments the convolution index. If the convolution index equals tlie 
number of bits in the convolution mask, the processor sets the convolution index to 0. The 
processor jimips to step n. 

9) The method of claims 7 or 8 where the decoding of the message bit from the sample in 
stq) m consists of reading least significant bit of the sample. 

10 ) The method of claims 7 or 8 wh^e the decoding of the message bit from the sample in 
step m consists of mapping a range of values onto a message bit For instance, if the 
sample value has a least significant tens digit of 0-4, then the message bit decoded is 0 and 
if the value has a least significant tens digit of 5-9, the message bit is 1 . 

11) 

tlie methods or claims 4 and 8 where tlie boolean truth table is TRUE for all inputs. Tliat is 
/(bitl,bit2) = TRUE for aU values of bitl and bit2 
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12) the methods of claims 3,4,5,6,7,8,9,10 and 11 where the samples are obtained from a 
sample stream representing digitized sound or music 

13) the method of claim 12 where the identical encode or decode process is performed on 
two sample streams representing channel A and channel B of digitized stereo sound 

14) tlie method of claim 12 where the sample streams representing channel A and channel 
B of a digitized stereo sound are interleaved before input to the stega-cipher apparatus and 
algorithm as a single sample stream and separated into two channels upon output 

15) the methods of claims 3,4,5,6,7,8.9,10 and 11 wh^e the samples are obtained from a 
sample stream represmting digitized video 

16) the methods of claims 3,4,5,6,7,8,9,10 and 11 where the samples are obtained from a 
sample stream representing a digitized image. 

17) 

tlie apparatus of claim 2 where said apparatus is contained in tamper-resistant packaging, 
which destroys circuitry and infomiation stored in said ^paratus if broken 

18) Hie method of claims 3 and 7 where a pre-encodmg step is used which furtlier 
customizes the message to be encoded by first calculating over which exact windows in the 
sample stream a message will be encoded, then computing a secure one way hash function 
of the samples in those windows, where the hash function is designed to be insensitive to 
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5 changes in the samples induced by the stega-cipher apparatus of claim 1 and tlie metliods 
of claims 3 and 7, and then including the resulting hash values in tlie message, which is 
signed and encrypted and finally encoded into the samples. 

This method is accomplished by 1) calculating the size of the message plus the added hash 
10 value, which adds a constant, predicatable number of bytes to the message 2) calculating 

the size of a signed mcrypted version of the message 3) processing the sample stream in a 
mamier which emulates the calculations performed in the method of claims 2 and 6 for the 
purpose of calculating hash values of each series of windows that will be used to encode 
the message, and creating a modified copy of ttie message which matches each particular 
15 series uniquely with each hash value, and 4) feeding this message to the apparatus of claim 
1 as decribed in claims 3 and 7. 

19) the methods of claims 3,4,5,6,7,8,9,10.11,12,13,14 and 18 where a trusted authority 
for the online distribution of content encodes the following information into a sample 
20 stream using the zqjparatus of claim 2 



the title 



the artist 



the copyright holder 



the body to which royalties should be paid 



.25 



general terms for publishers' distribution 



20) the method of claim 19 where a trusted authority combines the information listed in 



claim 19 and additionally a secure private key signed message from a publisher containing 



the following information 
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5 flie title 

the publisha-'s identification 
the terms of distribution 

any consideration paid for the right to distribute ttie content 
a brief statement of agreement wifli all terms Usted above 
10 and signs and encrypts this combined message using a pubKc key ciyptosystem and 
further encodes this signed and encrypted message into a sample stream 

21) the method of claim 20 where a publisher obtains the sample stream encoded in the 
claim 20 and additionally obtains tlie information listed in claim 19 from the authority and 
1 5 combines this with a message recieved from a consumer, which has been signed using a 
public key ciyptosystem and where the signed message contains the following information 
flie content title 
consume- identification 
the terms of distribution 
20 the consideration paid for the content 

a brief statement of agreement with the terms above 
and further, the publisher uses a public key cryptosystem to sign this combined 
information and fmaUy encodes the signed infonnation as a stega-cipher watemiark 

25 22) the method of claims 3,4.5,6.7.8.9 and 10 where the sample stream is obtained from at 
least one audio trad: contained within a digitized movie, video gae software, or other 
software 
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5 23) the method of claims 3,4,5,6,7,8,9 and 10 where the sample stream is obtained from at 
least one digitized movie or still image contained within video game or other software. 

24) the method of encoding information into content where the infomiation is contained in 
the differences between samples, rather than the value of an individual sample 

10 



53 



Appendix - Psuedo-code 



const int WINDOW.RESET = 256; 
const int WIND0W_SI2E = 128;- 
const int MARKER_BrTS = 1024; 
const int CHUNKLBfTS = 2048 * 8; 

int window, off set; 
int msg_bit_offset; 
int frequency_offset; 
Boolean useCell; 

/• 8 bits per bye, 1 byte per char V 
unsigned char frequency_mask[WIND0W_SIZE/8]; 
unsigned char window_mask[WINDOW_RESET/8]; 
uijsigned char msg_start_marker[MARKER_ BITS/8]; 
unsigned char msg_end.marker[MARKER_BITS/8]; 
Int 1 6 amplitude.sample.buff er[WIND0W_SI2E]; 
float power_frequency_buffer[WINDOW„SI2E]; 
unsigned char message_buffer[CHUNK_BITS/8]; 

void doFFT(Intl6 *amp_sannple_buffer. float *power_freq_buffer.int size); 

void dolnverseFFT(lntl 6 *amp_sample_buffer. float *power_freq_buff€r,int size); 

void initializeO; 

Bit getBit(unsigned char *buffer.int bitOffset); 

Boolean map(Bit window_bit, Bit band.bit, int window, int frequency); 
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5 Boolean getSamples(lntl 6 *amplitude_sample«buffer,int samples); 
void encodeO 

void initializeO 
{ 

10 /* message to be encoded is generated V 

/* message is prefixed with 1024 bit msg_start_ marker V 

/♦ message is suffixed with 1024 bit msg_end .marker V 

/* remaining space at end of message buffer padded with random bits V 

window_offset = 0; 

15 msg_bit_offset = 0; 

frequency_offset = 0; 
frequency.mask loaded 
window.mask loaded 
zeroAmpSampleBufferO; 

20 } . 

Boolean getSamples(Intl 6 *buffer,int samples) 
{ 

/* get samples number of samples and shift them contiguously into the sample 
25 buffer from right to left*/ 

if(samples < samples available) 
return false; 

else 

return true; 

30 } 
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void doFFT(lntl6 *sample_buffer. float *spectrum_buffer, int size) 
{ 

calculate FFT on sample_buffer, for size samples 
store result in spectrum buffer 

} 

void dolnverseFFT(lnt16 *sample_buffer.float *spectrum_buffer.int size) 
{ 

calculate inverse FFT on spectrum_buffer 
store result in sampe_buffer 

} 

Bit getBit(unsigned char *buffer,in bitOffset) 

{ 

returns value of specified bit in specified buffer 

either 0 or 1, could use Boolean (true/false) values for bit set of bit off 

} 

Boolean map(Bit window.bit.Bit band.bit.int window, int frequency. 
{ 

/♦ this is the function that makes the information difficult to find */ 
/♦ the inputs window_bit and band_bit depend only on the mask values 

used for encoding the information, they are 1 ) random, 2) secret •/ 
/* window and frequency values are used add time and frequency band dependent 

complexity to this function ♦/ 
/* this function is equivalent to a Boolean truth table with window * frequency * 

possible input combinations and 2 possible output V 
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/* for any input combination, the output is either true or false V 
/* window ranges from 0 to WINDOW.RESET -1 V 
/* frequency ranges from 0 to WIND0W_S!2E - 1 V 
return calculated truth value 

} 

void encodeBit(float *spectrum_buffer,int freq_offset,Bit theBit) 

{ 

/* modifies the value of the cell in spectrum.buffer, indexed by freq_offset 
in a manner that distinguishes each of the 2 possible values of theBit, 
1 or 0 

* / 

/* suggested method of setting the Least Significant bit of the cell == theBit */ 
/* alternative method of rounding the value of the cell upward or downward to 
certain fractional values proposed 
\ i.e. <= .5 fractional remainder signifies 0, > .5 fraction remainder 

signifies 1 

* / 

} 

void encodeQ 
{ 

initializeO; 
do { 

if(getSampfes(amplitude_sample_buffer) == false) 
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return 



doFFT(amplitude„sample_buffer,power_frequency.buffer.WINDOW_SIZE); 

for (frequency, off set 0; frequency .offset < WIND0W_SI2E; 
10 frequency_offset++){ 

useCell = map(getBit(window_mask,winclow„offset). 

getBit(frequency_maslc,frequency_offset). 
window.offset. frequency.offset); 

15 

if(useCeII == true){ 

encodeBit(power_frequency_buffer,frequency_offset, 

getBit(message_buffer,msg_bit_offset)); 
message_bit_offset ++; 
20 if(msg_bit_offset MESSAGEBITS){ 

^ initializeO; 

break; /* exit frequency loop */ 

} 

} 

25 } 

dolnverseFFT(amplitude_sample.buffer,power_frequency_buffer, 
WIN DOWNSIZE); 

30 outputSamples(amplitude_sample_buffer); 
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window_offset++; 



if(window_offset WINDOW_RESET){ 



window.offset 



= 0; 



10 



} while(true); 



The encodeO procedure processes an input sample stream using the specified frequency and 
15 window masks as well as a pre-formatted message to encode. 

encodeO processes the sample stream in windows of WINDOW.SIZE samples, contiguously 
distributed in the sample stream, so it advances WINDOW.SIZE samples at a time. 

20 For each sample window, encodeQ first compute the FFT of the window, yielding its Power 



Spectrum Estimation. For each of these window PSEs. encodeQ then uses the mapQ function to 
determine where in each PSE to encode the bits of the message, which it reads from the 
message buffer, on ebit at a time. Each time mapQ returns true, encode() consumes another 
sample from the message. 



After each window is encoded, encodeQ computes the inverse FFT on the PSE to generate a 
modified sample window, which is then output as the modified signal. It is important the sample 
windows NOT overlap in the sample stream, since this would potentially damage the preceeding 
encoding windows in the stream. 



25 



30 
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5 Once the message is entirely encoded, including its special end of message marker bit stream, 
encodeO resets it internal variables to begin encoding the message once more In the next 
window. encodeO proceeds in this manner until the input sample stream is exhausted. 

10 enum { 

Synchronizing, 
Locked 
}; /* decode states V 

15 unsigned char message_end_buffer[MARKER_BrTS]; 

Bit decodeBit(float *spectrum_buffer,int freq_offset) 



/* reads the value of the cell in spectrum_buffer, indexed by freq_offset 



20 



in a manner that distinguishes each of the 2 possible values of an 



encoded bit, 1 or 0 



* / 



/* suggested method of testing the Least Significant bit of the cell V 



/* alternative method of checking the value of the cell versus certain fractional 
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remainders proposed. 



i.e. <= .5 fractional remainder signifies 0, > .5 fraction remainder 



signifies 1 



♦ / 



return either 1 or 0 as appropriate 



30 
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Boolean decode() 
{ 

/* Initialization ♦/ 
state « Synchronizing 
window.offset = 0; 
set frequency nnask 
set window mask 
clear sample buffer 
int nextSamples = 1; 
int msg_start_offset = 0; 
clear message_end_buffer 
Bit aSit; 

Boolean bitsEqual; 
do{ 

if (state == Synchronizing)! 
nextSamples = 1 ; 
window_offset = 0; 

} 

else 

nextSamples = WINDOW.SIZE; 

if(getSamples(amplitude.sample_buffer) == false) 
return false; 

doFFT(amplitude_sample_buff€r,power_frequency_buffer, 
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WINDOW.SIZE);/* 2 V 



for (frequency_offset = 0; frequency_offset < WINDOW_SIZE; 
frequency .off set++){ 

useCell = map(getBit(window_mask,window_offset), 
getBit(frequency_mask, frequency, offset), 
window_offset, frequency.offset); 

if(useCell == true){ 

aBit ~ decodeBit(power.frequency_buffer, 

frequency, offset); 
setBit(message_buffer,message_bit.offset,aBit); 
message_bit_offset ++; 

} 

else 

continue; 
if(state == Synchronizing)} 
bItsEquat » 

compareBits(message_start_marker,message_buffer, 

message_bit.offset); 
if(!bltsEquaI){ 

message_bit.offset - 0; 

misaligned = true; 

break; /* exit frequency loop V 

} 

else if (message.bit.offset == MARKER_BrTS) 
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state == Locked; 

} 

else { 

/* locked onto encoded stream V 

shift aBit into right side of message_end_buffer 

bitsEqual = com pare Bits(message_end_buffer, 

msg_end_marker,MARKER_BITS); 
if (bitsEqual) 

return true; 

} 

} while (true); 

} 

The decodeO procedure scans an input sample stream using specified window and frequency 
masks, until it either decodes a valid message block, storing it in a message buffer, or 
exhausts the sample stream. 

The decodeO procedure starts in state Synchronizing, in which it does not know where in the 
sample stream the encoding windows are aligned. The procedure advances the sample window 
through the sample stream one sample at a time, performing the FFT calculation on each 
window, and attempting to decode valid message bits from the window. As it extracts each bit 
using the map() function, the decodeO procdure compares these bits against the start of 
message marker. As soon as a mismatch is detected, the decode() procedure knows it is not yet 
properly aligned to an encoding window, and immediately ceases decoding bits from the current 
window and moves to the next window, offset by 1 sample. The decode() procedure continues in 
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this manner until it matches successfully the complete bltstream of a start of message marker. 
At this point the decodeQ procedure assumes it is aligned to an encoded message and can then 
decode bits to the message buffer quickly, advancing the sample window fully at each 
iterations. It is now in Locked mode. For each bit it stores in the message buffer when in Locked 
mode, the decodeQ procedure also shifts the same bit value into the least significant bit of the 
message_end.buffer. After each bit is decoded in Locked mode, the decodeQ procedure checks 
compares the message„end_buffer with the msg_end_marker in a bit by bit manner. When a 
complete match is found, decodeQ is finished and returns true. If the sample stream is 
exhausted before this occurs, decodeQ returns false. If decodeQ returns true, a valid message 
is stored in the message buffer, including the start and end of message markers. 



64 



